Images are conventionally represented by a two-dimensional array of values in which each value represents a property of the image at a corresponding point on the image. In the case of gray-scale images, a single number representing the gradations of intensity from white to black, referred to as the gray scale, is stored. In the case of color images, each "value" is a vector whose components represent the gradations in intensity of the various primary colors, or some alternative color code, at the corresponding point in the image. A motion picture comprises a sequence of such images. Typically, thirty or more images are needed for each second of viewing time.
This representation of a motion picture corresponds to the output of a typical image-sensing device such as a television camera. Such a representation is convenient in that it is easily regenerated on a display device such as a CRT tube. However, the number of bits needed to represent the data is prohibitively large for many applications. A single 512.times.512 gray-scale image with 256 gray levels requires in excess of 256,000 bytes. At 30 frames per second, a communication channel with a bandwidth of approximately 64 million bits per second is needed to transmit the motion picture. A frill color 24 bit per pixel motion picture would require a bandwidth of more than 190 million bits per second.
This bandwidth is significantly greater than that available for many communications purposes. For example, high quality consumer telephone lines are typically limited to 64 thousand bits per second. Hence, commercially viable picture telephone systems would require some form of image compression.
Image compression systems used in motion picture compression applications make use of the redundancy between frames to reduce the amount of information that must be transmitted over the bandwidth-limited communication link. For example, many scenes in a motion picture include portions that are constant from frame to frame for several seconds, if not minutes. Prior art compression schemes attempt to divide the scene into the constant portion and the portion that changes from frame to frame.
The constant portion need be sent only once. Hence, if the constant portion comprises a significant fraction of the scene, considerable image compression can be realized. For example, if the entire scene were constant for 10 seconds, the information needed to reproduce the scene would be contained in one frame of the sequence and an indication of the number of times the frame is to be repeated. This would be approximately 1/300 of the information needed if the scene were sent without compression.
To better understand the manner in which this type of prior art image compression operates, consider a motion picture which is being sent over a bandwidth-limited communication link from a transmitter to a receiver. The receiver will be assumed to have a frame buffer which stores one frame of the motion picture. The contents of this frame are then displayed at the receiver. Ideally, the frame in the receiver will exactly match the corresponding frame at the transmitter.
Typically, the process is started by sending one complete frame of the sequence. This frame is referred to as the key frame. After the key frame is sent, successive frames are specified by sending information specifying changes that have occurred relative to the last frame displayed by the receiver. Typically, the difference between the current frame and the last frame is computed to generate an "incremental frame". If the pixels of the incremental frame are added to those of the key frame, the current frame is regenerated. Since most scenes change slowly, the incremental frames on average have much less information than a key frame; hence the incremental frames require less bandwidth to transmit than a key frame.
As the difference between the key frame and the succeeding frames increases, the incremental frame pixels increase in magnitude and the bandwidth required to send an incremental frame exceeds that of the communication channel. At this point, either a new key frame must be sent or only a portion of the incremental frame may be sent. In general, incremental coding schemes have not provided sufficient compression to allow motion pictures of the quality of commercial television to be sent over low-cost communication channels. Hence, further compression of the motion picture is typically required.
One method for reducing the amount of information that must be sent is to further code the key frame and successive incremental frames using sub-band coding techniques such as those used to compress two-dimensional still images. In these systems, each frame is filtered to generate a plurality of component images therefrom. The filtering process concentrates the information in the frame into one or more of the component images thereby reducing the number of bits that must be used to send the other component images. The degree of compression obtained using these techniques depends on the amount of spatially redundant information in the frames being compressed. Such schemes work well with the key frame, as this frame is a single two-dimensional image, and the schemes in question have been optimized for such images.
The degree to which the incremental frames may be compressed depends on the statistical distribution of the pixels in the frames. As the variance of this distribution increases, the bandwidth needed to transmit the frames increases. Hence, it would be advantageous to provide systems in which the variance of the pixel values in the frames is reduced.
Broadly, it is an object of the present invention to provide an improved apparatus and method for coding motion pictures.
It is a further object of the present invention to provide a method and apparatus in which the variances of the pixel distributions in the key frames and incremental frames is reduced relative to prior art systems.
These and other objects of the present invention will become apparent to those skilled in the art from the following detailed description of the invention and the accompanying drawings.